Image alignment method

ABSTRACT

An image alignment method includes: fetching at least two images; calculating a relative displacement between those adjacent images by utilizing a phase correlation algorithm; calculating an absolute displacement between any one of those images and the first image of those images; and computing a common area of those images by utilizing the relative displacement and the absolute displacement, then deleting remainder portions of the image excluding the common area. In the present invention, the phase correlation algorithm can be utilized to process numerous noise signals so as to get a higher precision of the image alignment.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to an image alignment method,and more particularly relates to an image alignment method utilizing aphase correlation method.

2. Description of the Prior Art

The microscopy technology has been evolved in many years and has made atremendous contribution in the development of technology. During therecent decade, the rapid development of high-performance personalcomputers further contributes to the maturity and the application of thetechnology. Additionally, using optical sectioning (tomography) anddigital image reconstruction to perform a three-dimensionalmicro-tomography has made a great impact on many fields.

In terms of a transmission electron microscope (TEM) and an X-raymicroscopy, objects need to be taken with the different projectionangles to produce the three-dimensional information and images, but allthese data need precise image alignment. And the displacements of theoriginal images is due to a mechanical shock or the defects, which areproduced when the images are taken from different angles or owing to thelocation movement of the machine caused by the thermal drift. In aconventional way, the manual operation is utilized to overcome theseissues, however, it is not only a waste of time but also easy to producehuman errors. An improved conventional study, the cross-correlationmethod, is used to resolve the manual problem, nevertheless, anotherimportant issue is aimed at the images taken with different angles arenot identical, which cannot be completely overcome.

SUMMARY OF THE INVENTION

In order to solve the aforementioned problems, one object of the presentinvention is to provide an image alignment method, wherein a phasecorrelation algorithm can be utilized to calculate the displacement oftwo images more accurately.

Another object of the present invention is to provide an image alignmentmethod, wherein it utilizes the phase correlation algorithm toreconstruct the images without fetching the same images.

Another object of the present invention is to provide an image alignmentmethod, wherein it utilizes the phase correlation algorithm to processmost of the noisy area so as to align the images more precisely.

Another object of the present invention is to provide an imagereconstruction method, wherein the misalignment problem due to thevibration and the wobble of the rotation stage in the tomography processcan be resolved by utilizing an image alignment method performed by thephase correlation algorithm.

A further object of the present invention is to provide an imagereconstruction method, wherein it utilizes an image alignment methodperformed by the phase correlation algorithm to resolve the photoquality problem due to the vibration during the digital image-capturedevice is operated.

Accordingly, one embodiment of the present invention provides an imagealignment method including: fetching at least two images; calculating arelative displacement between those adjacent images by utilizing a phasecorrelation algorithm; calculating an absolute displacement between anyone of those images and the first image of those images; and computing acommon area of those images by utilizing the relative displacement andthe absolute displacement, and deleting remainder portions of thoseimages excluding the common area.

Another embodiment of the present invention provides an imagereconstruction method including: fetching at least two images;calculating a relative displacement between those adjacent images byutilizing a phase correlation algorithm; calculating an absolutedisplacement between any one of those images and the first image ofthose images; computing a common area of those images by utilizing thoserelative displacement and those absolute displacement, and deletingremainder portions of those images exclusive of the common area;determining the rotation centers of those images; and reconstructing thethree-dimensional data of those images.

Another embodiment of the present invention provides an imagereconstruction method including: fetching at least two images;calculating a relative displacement between those adjacent images byutilizing a phase correlation algorithm; calculating an absolutedisplacement between any one of those images and the first image ofthose images; computing a common area of those images by utilizing thoserelative displacement and those absolute displacement, and deletingremainder portions of those images exclusive of the common area; andaccumulating the common area of every image.

Other advantages of the present invention will become apparent from thefollowing description taken in conjunction with the accompanyingdrawings wherein are set forth, by way of illustration and example,certain embodiments of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the accompanying advantages of thisinvention will become more readily appreciated as the same becomesbetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flow chart of the image alignment method according to oneembodiment of the present invention;

FIG. 2 is a flow chart of the image reconstruction method according tothe first embodiment of the present invention; and

FIG. 3 is a flow chart of the image reconstruction method according tothe second embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The specific examples of the invention are described in detail asfollow. However, it should be understood that the invention is not to belimited to the particular form disclosed herein.

FIG. 1 is a flow chart of the image alignment method to illustrate oneembodiment of the present invention. Such as shown in FIG. 1, first,fetching at least two images p_(i) and p_(i+1) from the taken pictures(S10) where p_(i) denotes the ith image, i=1 to N, and N is a positiveinteger which denotes the number of images, wherein, according todifferent applications, these pictures of the same object can be shot atthe same angle or different angles. Going on the next step, calculatinga relative displacement between any two consecutive images p_(i) andp_(i+1) (S20), wherein the relative displacement can be estimated byutilizing a phase correlation algorithm. Further, calculating anabsolute displacement between any one of these images and the firstimage (S30), such as calculating a position shift between p₁ and p_(i)where p₁ denotes the 1^(st) image and p_(i) denotes the ith image whichcould be any one of the images, wherein the absolute displacement can becomputed by a phase correlation algorithm. In another embodiment, theabsolute displacement also can be figured out by accumulating therelative displacement between every two consecutive images p_(i) andp_(i+1) between the image p₁ and p_(i+1) to obtain the position shiftbetween p₁ and p_(i+1). Finally, a common area of the whole images canbe worked out by utilizing the relative displacement and the absolutedisplacement, and then the remainder portions of these images excludingthe common area is deleted (S40).

Continuing the above description, before calculating the relativedisplacement or the absolute displacement, the aforementioned methodfurther includes performing an image preprocessing operation for theseimages. In one embodiment, the image preprocessing operation includesany one of a sharpening process, a smoothing process, and a noiseremoval process. Wherein the image preprocessing operation is conducedto process some unnecessary noise of these images, or probably toenhance the signal of these images to increase the accuracy of thefollowing image alignment or even the accuracy of the imagereconstruction.

In one embodiment, the method of performing the forementioned relativedisplacement or the absolute displacement utilizes a Fourier transformmethod, a fast Fourier transform method, or the operation of the Fouriertransform method and the fast Fourier transform. The operation method isshown as equation (1) in the following: first, performing the Fouriertransform of two images, such as transforming the image p1 and image p2to two Fourier transform values, F[p1] and F[p2]; next, calculating thecorrelation of these two images, i.e., multiplying one Fourier transformvalue F[p1] by the complex conjugate of another Fourier transform valueF[p2], such as the mathematic expression F[p1](F[p2])*; more, dividingthe product computed above by the modului of these two images, such asthe mathematic expression

$\frac{{F\left\lbrack {p\; 1} \right\rbrack}\left( {F\left\lbrack {p\; 2} \right\rbrack} \right)^{*}}{{{F\left\lbrack {p\; 1} \right\rbrack}}{{F\left\lbrack {p\; 2} \right\rbrack}}};$furthermore, multiplying a spatial filter, such as G, in the presentembodiment, the spatial filter is a low-pass filter; then, performingthe inverted Fourier Transform so as to find the maximum value in thespatial coordinates, which is the shift of these two images.

$\begin{matrix}{\left\lbrack {{p\; 1},{p\; 2}} \right\rbrack_{{shift}{\lbrack{x,y}\rbrack}} = {{Max}\left\{ {{F^{- 1}\left\lbrack \frac{{F\left\lbrack {p\; 1} \right\rbrack}\left( {F\left\lbrack {p\; 2} \right\rbrack} \right)^{*}}{{{F\left\lbrack {p\; 1} \right\rbrack}}{{F\left\lbrack {p\; 2} \right\rbrack}}} \right\rbrack}G} \right\}}} & {{equation}\mspace{14mu}(I)}\end{matrix}$

Wherein

-   -   p1: one image    -   p2: another image;    -   G filter; and    -   x,y: x denotes the x-coordinate of the maximum displacement of        p1 and p2; and y denotes the y-coordinate of the maximum        displacement of p1 and p2.

In the meantime, this image alignment method can be applied to differentembodiments of image reconstruction method are described.

FIG. 2 is a flow chart of the image reconstruction method to illustrateone embodiment of the present invention. In the embodiment, the imagealignment method mentioned above is applied to the micro-tomography.Such as shown in FIG. 2, first, fetching at least two images from thesetaken pictures (S10), wherein these pictures are taken from the sameobject and shot at different angles. Going on the next step, calculatinga relative displacement between these adjacent images (S20), wherein therelative displacement can be performed by utilizing a phase correlationalgorithm, such as a Fourier transform, a fast Fourier transform, or theoperation of these two methods. Further, calculating an absolutedisplacement between any one of these images and the first image (S30),that is, based on the first image, calculating the displacements betweenevery one image and first image (called the absolute displacement). Inone embodiment, the operation of the absolute displacement is alsoperformed by the phase correction algorithm. Next, estimating a commonarea of the whole images by utilizing the relative displacement and theabsolute displacement, and then deleting remainder portions of theseimages excluding the common area (S40). In the meantime, the process ofthe image alignment is completed. Next, determining the rotation centersof those images (S50). Finally, reconstructing the three-dimensionaldata of the image (S60).

Continuing the above description, before calculating the relativedisplacement or the absolute displacement, the aforementioned methodfurther includes performing an image preprocessing operation for theseimages. In one embodiment, the image preprocessing operation includesany one of a sharpening process, a smoothing process, and a noiseremoval process. In one embodiment, the method of determining therotation centers is to decide the rotation sinograms of the images, butit is not limited to the one described above.

In one embodiment, further including interpolating or extrapolating theimages before reconstructing the three-dimensional data. Wherein theimages are interpolated or extrapolated to set the pixel number to2^(k), and k is a positive integer. Next, a filtered-back-projection(FBP) method based on Fourier slice theorem can be performed toreconstruct the three-dimensional data. However, it is not limited tothe interpolation or extrapolation of data to set the pixel number to2^(k). In another embodiment, method of paddling zero to the data to setthe pixel number to 2^(k) can be worked too.

Referring to FIG. 3, FIG. 3 is a flow chart of the image reconstructionmethod to illustrate the second embodiment of the present invention. Inthe embodiment, the aforementioned image alignment method is applied onthe steady shot function of the digital image-capture device such as adigital camera for the continuous shooting. First, changing the exposuretime t to t/N. In the predetermined exposure time, fetching N or lessthan N pictures, wherein N is a positive integer. Such as shown in thefigure, the steps (such as S10, S20, S30, and S40) before calculatingthe common area of the image are the same as the aforementioned imagereconstruction, it does not give unnecessary details here. After thecommon area is working out, accumulating the common areas of everyimages to enhance the signal-to-noise ratio (SNR). This method is toshorten the exposure time so as to fetch the highly noisy but clearimage. These images can be processed by an image preprocessing methodthat includes any one of a sharpening process, a smoothing process, anda noise removal process. After image preprocessing, aligning the imagesby the phase correlation algorithm, then accumulating the image of thecommon area to enhance the SNR so as to perform the steady shot functionof digital image-capture device with short exposure time and clearimage.

On consideration for the restriction of the data, central processingunit, and memory of the digital image-capture device itself, the imagefetching method can be fetching two images which can be aligned inserial; calculating the common area; and adding the computed common areato one image, and deleting another image to save the memory space. Afterthese processes, fetching next image to perform the same processes. Bythis way, the common area of any two successive images can beaccumulated to the last one so as to reconstruct the image.

Accordingly, one of features is to utilize the phase correlationalgorithm to calculate the displacement of images to effectively performthe image alignment. The image alignment method can be applied in thepreprocessing of other image reconstruction methods, such as theelectron microscope, the X-ray microscope, the tomography, or themicro-tomography. Furthermore, owing to the relative vibration betweenthe digital image-capture device and the photographed object or becausethe photographed object moves in a high speed, the image alignmentmethod also can be applied on the digital image-capture device, such asdigital cameras or mobile camera phones. Additionally, the applicationof the method is not limited to the aforementioned ones; it can beapplied in other system that needs image alignment process.

To sum up the forgoing descriptions, the present invention utilizes animage alignment method using a phase correlation algorithm to fast andprecisely perform the operation for the displacement of two images. Inaddition, the image alignment method can be utilized to resolve theimage alignment problem with similar images. More, the phase correlationalgorithm can be used to process most of the noisy area so as to alignthe images more precisely. Further, the misalignment problem due to thevibration and wobble of the rotation stage in the tomography process canbe resolved by utilizing an image alignment method performed by thephase correlation algorithm. Furthermore, the image alignment method isperformed by the phase correlation algorithm to resolve the photoquality problem due to the vibration when the digital image-capturedevice is operated.

While the present invention is susceptible to various modifications andalternative forms, a specific example thereof has been shown in thedrawings and is herein described in detail. It should be understood,however, that the invention is not to be limited to the particular formdisclosed, but to the contrary, the invention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the appended claims.

1. An image alignment method comprising: fetching at least twoconsecutive images from images captured from an object sequentially;calculating a relative displacement between said two consecutive images,wherein said relative displacement is a shift between any twoconsecutive images of said images which can be calculated by a phasecorrelation algorithm utilizing a filter G to pass low-frequencysignals, and is estimated by the following equation:$\left\lbrack {{p\; 1},{p\; 2}} \right\rbrack_{{shift}{\lbrack{x,y}\rbrack}} = {{Max}\left\{ {{F^{- 1}\left\lbrack \frac{{F\left\lbrack {p\; 1} \right\rbrack}\left( {F\left\lbrack {p\; 2} \right\rbrack} \right)^{*}}{{{F\left\lbrack {p\; 1} \right\rbrack}}{{F\left\lbrack {p\; 2} \right\rbrack}}} \right\rbrack}G} \right\}}$where p¹ and p² are any two consecutive images of said images, F is aFourier transfrom function, G is said filter, x denotes the x-coordinateof the maximum displacement of p¹ and p², and y denotes the y-coordinateof the maximum displacement of p¹ and p²; calculating an absolutedisplacement which is a shift between any one of said images and thefirst image of said images; and computing a common area of said imagesby utilizing said relative displacement and said absolute displacement,and deleting remainder portions of said images excluding said commonarea.
 2. The image alignment method according to claim 1, furthercomprising performing an image preprocessing operation for said imagesbefore calculating said relative displacement or said absolutedisplacement.
 3. The image alignment method according to claim 2,wherein said image preprocessing operation comprises any one of asharpening process, a smoothing process, and a noise removal process. 4.The image alignment method according to claim 1, wherein said absolutedisplacement is calculated by utilizing a phase correlation algorithm.5. The image alignment method according to claim 1, wherein said Fouriertransform function is a fast Fourier transform function.